Anonymity as an instructional scaffold in peer assessment : its effects on peer feedback quality and evolution in students’ perceptions about peer assessment skills

This is a post-peer-review, pre-copyedit version of an article published in European Journal of Psychology of Education 33.1 (2018): 75-99. The final authenticated version is available online at: http://dx.doi.org/10.1007/s10212-017-0339-8Although previous research has indicated that providing anonymity is an effective way to create a safe peer assessment setting, continuously ensuring anonymity prevents students from experiencing genuine two-way interactive feedback dialogues. The present study investigated how installing a transitional approach from an anonymous to a non-anonymous peer assessment setting can overcome this problem. A total of 46 bachelor’s degree students in Educational Studies participated in multiple peer assessment cycles in which groups of students assessed each other’s work. Both students’ evolution in peer feedback quality as well as their perceptions were measured. The content analysis of the peer feedback messages revealed that the quality of peer feedback increased in the anonymous phase, and that over time, the feedback in the consecutive non-anonymous sessions was of similar quality. The results also indicate that the transitional approach does not hinder the perceived growth in peer feedback skills, nor does it have a negative impact on their general conceptions towards peer assessment. Furthermore, students clearly differentiated between their attributed importance of anonymity and their view on the usefulness of a transitional approach. The findings suggest that anonymity can be a valuable scaffold to ease students’ importance level towards anonymity and their associated need for practiceThe first author’s research was funded by Ghent University BOF fund number BOF13/24J/115. The second author’s research was funded by the Spanish Ramón y Cajal program number RYC-2013-1346

'Now you know what you’re doing right and wrong!' Peer feedback quality in synchronous peer assessment in secondary education

This study explores the effects of peer assessment (PA) practice on peer feedback (PF) quality of 11th grade secondary education students (N= 36). The PA setting was synchronous: anonymous assessors gave immediate PF using mobile response technology during 10 feedback occasions. The design was quasi-experimental (experimental vs. control condition) in which students in one condition received a scaffold to filter out relevant information they received. It was expected that this filter-out scaffold would influence PF quality in subsequent tasks in which they were assessors. PF content analysis showed that offering multiple PF occasions improved PF quality: messages contained more negative verifications and informative and suggestive elaborations after the intervention. However, no effects were found of filtering out relevant information on PF quality. Moreover, students’ perceived peer feedback skills improved which was in correspondence with their actual quality improvement over time. Additionally, the perceived usefulness of the received feedback was rated high by all participants

Teachers’ reasons for using peer assessment: positive experience predicts use

The final publication is available at Springer via http://dx.doi.org/10.1007/s10212-015-0282-5Peer assessment (PA) is one of the central principles of formative assessment and assessment for learning (AfL) fields. There is ample empirical evidence as to the benefits for students’ learning when AfL principles are implemented. However, teachers play a critical role in mediating the implementation of intended policies. Hence, their experiences, beliefs, and attitudes towards PA are important factors in determining whether the policy is actually carried out. A survey of over 1500 primary, secondary, and higher education teachers in Spain elicited their beliefs and values around PA as well as other aspects of formative assessment; only 751 teachers provided complete responses to all PA items. Teachers reported occasional use of PA in their classrooms but with positive experience of it. The vast majority did not use anonymous forms of PA and half of the teachers considered the students were accurate when assessing peers. Confirmatory factor analysis and structural equation modeling were used to examine relationships of attitudes and beliefs to self-reported frequency of using of PA. The self-reported frequency of using PA was strongly predicted by teacher experience of PA which included positive reasons for using PA, rather than negative obstacles for avoiding, prior use, and beliefs that students should participate in assessment, and willingness to include PA in gradingResearch funded by personal grant to Ernesto Panadero under Ramón y Cajal framewok (RYC-2013-13469

Assessment in Bologna context from the teaching perspective, similarities and differences between disciplines

In higher education, assessment is especially significant due to the high level of autonomy and self-regulation that is assumed in the students at this stage. While there is a bulk of research on which assessment evidences are used in higher education, research on how university professors design these evidences is lacking. Using a mixed method technique, we analyzed the assessment methodologies used in three different degrees (Mathematics, Medicine and Sport Sciences), and the design process followed by the teachers in each degree. We found important differences in the assessment methodologies used and the approaches to the assessment design in the degrees. This study shows the way in which teachers of different degrees modified their assessment methods during the Bologna process, as well as the factors that influenced them throughout the process

¿Cómo autorregulan nuestros alumnos?: Modelo de Zimmerman sobre autorregulación del aprendizaje

El uso de las estrategias de aprendizaje explica de forma crucial los resultados académicos de los alumnos. Un buen uso de estas estrategias permite un aprendizaje profundo y transferible. Los modelos de autorregulación se basan en teorías holísticas y comprehensivas sobre las estrategias de aprendizaje. En este trabajo se analiza el modelo de Zimmerman (2000; 2003; Zimmerman y Moylan, 2009), detallando los diferentes procesos recogidos en el modelo con ejemplos para ayudar a su visualización. El modelo de Zimmerman es uno de los más extendidos en la literatura científica en el campo siendo ampliamente citado por los especialistas en el campo. El trabajo de Zimmerman parte de una base socio-cognitiva, con especial cobertura de las influencias mutuas entre motivación y autorregulación. En este trabajo también se comparan otras teorías sobre aprendizaje autorregulado con la de Zimmerman para explicar el concepto de una forma más completaThe use of learning strategies is crucial for students‟ academic performance and promoting deeper learning approaches. The self-regulated learning models offer comprehensive theoretical backgrounds. These enable more holistic approaches to the use of learning strategies. In this paper, Zimmerman‟s (2000; 2003; Zimmerman & Moylan, 2009) cyclical model of self-regulated learning is described and analysed as one of the most comprehensive. The model is grounded in social cognitive theory and is comprised of three phases (forethought, performance and self-reflection) with a special focus on the influences of motivation on self-regulation. The differ-ent processes included in the model are analysed here in detail. Zimmerman‟s framework is considered in relation to other self-regulated learning models in order to recognize its importance in theory and practiceThis research was funded via fellowship awarded to Ernesto Panadero by the Spanish Education Department (Ref. SEJ2005-00994) and Alianza 4 Universidade

Self-assessment effects review: empirical evidence about its implementation using self-grading, rubrics and scripts

Universidad de VigoEsta revisión analiza trabajos sobre autoevaluación agrupados en: (1) autocalificación, o autoevaluación sin criterios de evaluación, (2) autoevaluación usando rúbricas y (3) autoevaluación usando guiones. Tras una búsqueda bibliográfica, se realizó un análisis narrativo del contenido de los diferentes estudios seleccionados. Con respecto a la autoevaluación sin criterios de evaluación destaca la falta de uniformidad en los resultados obtenidos por los distintos estudios aunque, en general muestran que la misma no garantiza un aprendizaje profundo de los alumnos. El uso de rúbricas y guiones presenta mejores efectos en el aprendizaje pero quedan aspectos por investigar: los estudios con rúbricas tienen diseños empíricos débiles y los estudios con guiones no han sido realizados en contextos reales. Las conclusiones indican que ofrece más garantías el uso de rúbricas y guiones. Se señalan líneas futuras de investigaciónThis review analyzes studies on the effects of self-assessment. These studies are grouped as: (1) self-grading, or interventions to promote self-assessment without assessment criteria, (2) use of rubrics, and (3) use of scripts. A literature search was conducted followed by a narrative content analysis of the selected studies. Regarding selfassessment without assessment criteria, the results are heterogeneous, though, in general, they show that these interventions do not warrant an improvement in the students’ learning. The use of rubrics and scripts presents more promising effects, but there are still important aspects to explore: studies on rubrics tend to rely on weak research designs, while studies on scripts, in general, have not been implemented in real classrooms settings. The main conclusion is that rubrics and scripts offer better conditions for self-assessment. Future lines of research are discusse

Instructional Help for Self-assessment and Self-regulation: Evaluation of the Efficacy of Self-assessment Scripts vs. Rubrics

Tesis doctoral inédita leída en la Universidad Autónoma de Madrid, Facultad de Psicología. Departamento de Psicología Básica. Fecha de lectura: 25 de marzo de 201

Third wave of measurement in the self-regulated learning field: when measurement and intervention come hand in hand

This is an Accepted Manuscript of an article published by Taylor & Francis Group in Scandinavian Journal of Educational Research on 24/07/2015, available online:http://www.tandfonline.com/10.1080/00313831.2015.1066436Measurement is a central issue for the self-regulated learning (SRL) field as SRL is a phenomenon difficult to measure in a reliable and valid way. Here, 3 waves in the history of SRL measurement are identified and profiled. Our focus lies on the third and newest one, which combines measurement and intervention within the same tools. The basis for this approach is located in the reactivity principle via students’ self-monitoring: when students are aware of their actions, they can react and change what is needed. That happens when the measurement tools promote students’ self-monitoring which turn part of the intervention then. Examples of this new approach to SRL measurement and guidelines for implementing it are presentedThe first author was funded via Spanish Ministerio de Economía y Competitividad, programa Ramón y Cajal [RYC-2013-13469]. The first and third author were funded via the Finnish Academy, project name PROSPECTS (PI: Sanna Järvelä, [grant number: 24301274]

Peer assessment use, its social nature challenges and perceived educational value : a teachers’ survey study

The implementation of peer assessment (PA) in the classroom faces considerable interpersonal challenges. In this quantitative survey study (N = 225) we focus on the current use and format of PA among secondary education teachers in Flanders and explore teachers’ awareness of these interpersonal challenges. We validated an instrument for measuring teachers’ awareness which was then used to investigate how this awareness level relates to their conceptions of the educational value of PA. SEM results show that teachers are slightly to moderately aware with regard to their students’ concerns about the impact of interpersonal processes in PA as well as the importance students attribute to anonymity within PA. This study illustrates that teachers’ perceived accuracy of PA is a major predictor of their belief in its educational value and opens up a new avenue for research on teachers’ awareness of interpersonal processes in PA

Rubrics vs. self-assessment scripts effect on self-regulation, performance and self-efficacy in pre-service teachers

This is the author’s version of a work that was accepted for publication in Studies in Educational Evaluation. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in Studies in Educational Evaluation, 39, 3 (2013) DOI 10.1016/j.stueduc.2013.04.001Two approaches to self-assessment are optimal, because they include the assessment criteria: rubrics and scripts. The aim of this study is to compare the effect of rubrics and scripts on self-regulation, performance and self-efficacy. A total of 69 pre-service teachers participated in the study. During a semester the participants were trained to design multimedia material in three experimental conditions (rubrics, scripts and control). Results showed that students using the scripts had higher levels of learning self-regulation after the intervention, whereas rubrics decreased performance/avoidance self-regulation (negative self-regulatory actions detrimental to learning). No significant effects were found for students’ performance or self-efficacy. Students preferred the use of rubrics to the use of scriptsSupport for this research was provided by grants from the Spanish Education Department to Ernesto Panadero (Ref. SEJ2005-00994) and to Jesús Alonso-Tapia (EDU2009-11765